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Figure  1 .  Principles  used  to  design for  automation  also  simplify  human  assembly .  (Adapted 
from  Helander  &  Nagamachi,  1992) 
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Ouring  the  1980s,  General 
Motors  invested  $80  bil¬ 
lion  in  automated  manu¬ 
facturing.  Automation  and  robotics 
were  fairly  new  as  a  concept,  and  it 
was  seen  by  U.S.  industry  as  the 
savior  of  competitiveness.  According 
to  an  article  in  The  Economist  (August 
10,  199D  about  $20  billion  was  lost 
since  the  automation  did  not  prove 
entirely  effective.  Most  other  large  in¬ 
dustries  had  similar  experiences;  losses 
and  early  write-offs  were  often 
greater  than  25%.  At  the  time,  there 
was  over-reliance  on  technology. 


Many  implementation  problems,  such 
as  task  allocation,  training,  vertical  and 
horizontal  task  expansion,  were  not 
successfully  addressed. 

The  introduction  of  robots  for  as¬ 
sembly  had  an  interesting  effect  on 
product  design.  Robots  are  typically 
not  equipped  with  vision  (because  it  is 
too  expensive)  and  they  have  poor 
finger  dexterity.  To  make  assembly 
possible,  product  design  had  to  con¬ 
sider  many  special  requirements  such 
as  “Incorporate  washers  with  screws,” 
“Use  chamfers  on  holes  and  pins  to 
Continued  on  page  2 
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increase  the  target  area,”  “Use  sym¬ 
metrical  parts,  so  that  they  don’t  have 
to  be  turned  around,”  “Use,  at  most, 
two  sizes  of  screws,”  and  “Use  snap 
and  insert  assembly.” 

As  a  result,  product  design  changed 
radically.  It  came  somewhat  as  a  sur¬ 
prise  that  the  identical  changes  also 
greatly  facilitated  manual  assembly — 
to  the  extent  that  there  was  no  true 
advantage  in  using  robots  (see  Fig.  1). 
The  great  irony  of  this  development  is 
that  it  took  robots  to  realize  that  hu¬ 
man  assembly  work  can  be  simplified. 

Product  designs  create  jobs  on 
the  manufacturing  floor,  and  there 
are  now  opportunities  to  design  prod¬ 
ucts  that  are  easy  to  assemble,  im¬ 
prove  work  posture,  reduce  product 
weight  and  ease  manual  handling, 
introduce  a  variety  of  work  responsi¬ 
bilities,  reduce  safety  hazards,  and 
maybe  even  improve  job  satisfaction 
(Helander&Furtado,  1992;  Nagamachi 
&  Yamada,  1992). 

Today,  manufacturing  depends  on 
machinery  and  automation  for  many 
processes  such  as  surface  treatment, 
milling,  and  cutting.  However,  when 
it  comes  to  assembly  there  is  not 
necessarily  an  advantage.  The  versa¬ 
tility  and  adaptability  of  human  labor 
is  so  great  that  automation  does  not 
always  pay  off.  A  printer  assembly  line 
at  the  IBM  plant  in  Charlotte  serves  to 
|  illustrate  the  point.  In  1984,  after  sev¬ 
eral  product  modifications,  it  was 
changed  from  an  automated  system  to 
a  manual  labor  one. 

Computer-Aided  Design 
and  Computer-Integrated 
Manufacturing 

Despite  our  reservations  concern¬ 
ing  automation,  computers  remain 
I  useful  for  product  design  and  manu¬ 
facturing  planning.  Computer-aided 
design  (CAD)  terminals  were  first  used 
as  substitutes  for  drawing  boards,  and 
this  application  has  proven  very  use¬ 
ful.  Elements  of  a  drawing  can  also  be 
subjected  to  computer-aided  engineer¬ 
ing  (CAE)  analysis  (e.g.,  finite  element 
analysis)  and  instructions  for  manu- 
'  facturing  that  are  implicit  from  a  draw¬ 
ing  can  be  transferred  to  numerical 
I  control  (NC)  machines  and  robots. 

In  concurrent  engineering,  computer 
modeling  of  manufacturing  has  be¬ 


come  even  more  strategic.  It  used  to 
be  that  a  product  designer  would 
finish  a  design  and  then  hand  it  over  to 
a  process  designer  who  would  design 
the  manufacturing  process.  Such  a 
step-wise  procedure  produces  a  sub- 
optimal  solution.  In  concurrent  engi¬ 
neering,  product  and  process  design¬ 
ers  jointly  design  a  product  satisficing 
several  criteria  (see  Table  1).  This 
saves  time  and  reduces  costs. 

With  this  scenario,  the  knowledge 
base  for  design  must  expand — to  the 
extent  that  corporate  strategic  plan¬ 
ning  is  affected.  Design  must  involve 
a  variety  of  professionals  with  CAD 
being  transformed  into  Computer 
Supported  Cooperative  Work  (CSCW). 
But  there  is  an  alternative  scenario: 
my  colleague,  Roland  Ortengren  (1992) 
predicted  that  90%  of  ergonomic  re¬ 
quirements  in  product  design  can  be 
handled  by  individuals  without  pro¬ 
fessional  training  in  ergonomics.  If  so, 
maybe  a  single  but  resourceful  “super 
designer”  could  do  the  job  guided  by 
expert  systems. 

The  use  of  CSCW  brings  many  in¬ 
triguing  questions  related  to  commu¬ 
nication  style  and  collaboration.  How 
can  individuals  negotiate  and  trade-off 
design  solutions?  Maybe  one  can  rely 


on  computer  support!  For  example, 
Questions,  Options,  and  Criteria  (QOC) 
(MacLean,  Young,  Bellotti,  &  Moran, 
1991)  is  a  software  system  which  docu¬ 
ments  design  criteria,  options  for  de¬ 
sign,  and  solutions. 

Computer-integrated  manufacturing 
(CIM)  is  difficult  to  implement. 
Siemieniuch  (1992)  told  the  horror 
story  of  a  £9  million  project  which  set 
out  to  design  a  “user-friendly”  CIM 
system.  But  the  project  failed,  since 
the  various  pieces  of  software  were 
not  well  integrated.  A  critical  element 
in  this  failure  was  the  lack  of  strategic 
planning  and  testing  of  typical  work 
activity  scenarios. 

In  proposing  a  solution  to  integra¬ 
tion  of  software,  there  is  a  fundamen¬ 
tal  problem.  There  is  not  yet  a  credible 
theoretical  framework  for  task  alloca¬ 
tion  in  human-computer  interaction. 
Upmann  (1992)  suggested  that  tasks 
demanding  and  promoting  human 
skills,  such  as  planning,  assessment, 
development  of  solution  strategies, 
heuristics,  and  rational  thinking,  should 
be  reserved  for  the  user.  The  com¬ 
puter  should  relieve  the  user  of  rou¬ 
tine  work  such  as  “ordering,”  “search¬ 
ing  for,”  and  “out-putting  data.”  This 
proposal  may  be  too  simplistic,  how- 


Table  1. 

Criteria  in  Concurrent  Engineering 

■  Ease  of  assembly 

■  Low-cost  manufacturing  process 

■  Product  maintenance 

■  Visual  inspection 

■  Worker  and  user  safety 

■  Product  usability  and  aesthetics 

■  Packaging  and  transportation 

■  Clean  manufacturing 

■  Reusability  of  parts  and  disassembly 

■  Parts  commonality 
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Top-Down 
Planning  of 
Manufacturing 


Bottom-Up 
Evaluation  of  Design 
and  Iteration 


Human  Factors  Issues 

and  Research  Problems 

■  Organizational  design 

■  Macro-ergonomics 

■  Corporate  policies  for 
product  and  process  design 

■  Criteria  for  concurrent 
engineering 

■  How  do  designers  design? 

■  Team  work  and  computer- 
supported  cooperative  work 

■  Computer  interface  design 

■  Criteria  for  allocation  of  tasks 

■  Cognitive  workload 

■  Workstation  design 


i  Measures  of  productivity 
and  job  satisfaction 


Figure  2.  Manufacturing  may  be  configured  as  a  feedback  system  with  top-down  planning  and  bottom-up  evaluation. 


ever,  since  computers  can  overcome 
many  of  the  limitations  and  biases  that 
humans  display  in  decision  making. 
In  addition,  Upmann’s  suggestion  does 
not  solve  the  problem  of  “fluency”  in 
dialogue  or  interchange  of  knowledge 
between  the  computer  and  user.  There 
are  many  fundamental  cognitive  is¬ 
sues  in  human-computer  communica¬ 
tion:  style  of  reasoning  and  critiquing, 
sharing  of  goals  and  partial  solutions, 
development  of  mutual  responsibili¬ 
ties  in  command-control,  and 
adaptivity  (who  is  the  leader — and 
when?).  Moreover,  a  valid  nomencla¬ 
ture  of  human-computer  task  alloca¬ 
tion  is  still  missing,  and  experimental 
evidence  is  far  away. 

Organizational  Impact 

CIM  is  also  used  for  day-to-day  plan¬ 
ning  in  manufacturing.  In  modern 
small-batch  manufacturing,  there  are 
needs  for  computational  planning  of 
resources  and  scheduling  of  jobs 
among  machines.  (Actually  both  of 
these  issues  could  also  be  incorpo¬ 
rated  in  concurrent  engineering.)  Typi¬ 
cally,  computational  planning  is 
handled  by  a  system  such  as  Manufac¬ 


turing  Resource  Planning  (MRP)  II 
(Rindom,  1992).  Modern,  small-batch 
manufacturing  requires  continuous  de¬ 
cision  making  based  on  feedback  from 
the  operation  itself — rather  than  the 
typical  feed-forward  which  is  implied 
by  management  orders,  centralized 
plans,  and  schedules.  These  compu¬ 
tations  should  not  necessarily  be  per¬ 
formed  by  an  engineer  or  supervisor, 
however.  In  this  turbulent  environ¬ 
ment,  Rasmussen  (1992),  for  instance, 
has  asserted  that  planning  and  control 
are  best  exercised  from  the  shop  floor. 
Put  a  computer  on  the  shop  floor,  and 
the  worker  will  do  it.  Thus  the  intro¬ 
duction  of  computer  support  has  im¬ 
plications  for  manpower  planning  and 
decentralization  of  decision  making. 
The  greater  the  uncertainty  in  plan¬ 
ning,  the  greater  is  the  organization’s 
need  to  rapidly  respond  to  change. 
This  requires  a  high  degree  of  profes¬ 
sionalism,  independence,  and  flexibil¬ 
ity  of  employees,  particularly  those  on 
the  shop  floor. 

The  Need  for  Research 

Figure  2  summarizes  many  of  the 
needs  for  human  factors  research  that 


are  implicit  from  this  article. 

The  left  portion  of  Figure  2  illus¬ 
trates  that  product  design  carries  im¬ 
plications  for  allocation  of  resources, 
including  task  allocation  between 
agents.  The  design  is  then  assessed 
with  respect  to  a  multitude  of  concur¬ 
rent  criteria  (in  Fig.  2  just  two  are 
noted:  productivity  and  job  satisfac¬ 
tion).  If  the  outcome  of  the  assess¬ 
ment  is  unsatisfactory,  there  is  a  de¬ 
sign  iteration.  Product  design  is 
changed,  which  changes  task  alloca¬ 
tion,  and  so  forth.  When  all  criteria 
have  been  satisficed  there  are  straight¬ 
forward  implications  for  design  of 
work-stations  and  processes.  In  the 
right  portion  of  Figure  2  several  re¬ 
search  issues  are  noted. 

As  DeKeyser  (1992)  has  observed, 
there  is  a  great  need  for  field  studies  in 
automated  manufacturing.  Highly  per¬ 
tinent  cognitive  and  socio-technical 
issues  cannot  be  studied  well  in  the 
laboratory  environment.  Parsons  (1992) 
noted  that  since  the  Hawthorne  re¬ 
search  at  the  Western  Electric  plant 
around  1935,  no  other  field  study  in 
the  U.S.  has  been  as  pioneering  and 
influential.  He  asked,  “Is  something 
Continued  on  page  4 
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wrong  with  American  industry  that 
more  than  five  decades  have  passed 
without  a  research  enterprise  equiva¬ 
lent  in  scope  to  investigate  the  impact 
of  new  automation  and  technology?” 
Given  the  tremendous  influence  that 
these  capabilities  have  already  had 
on  manufacturing  processes  and 
the  marked,  ever  increasing  power  to 
bring  about  even  greater  change, 
it  seems  the  time  is  right  to  initiate 
such  efforts.# 

Martin  Helander,  PhD .,  is  a  Professor  of 
Industrial  Engineering  at  the  State  Univer¬ 
sity  of  New  York,  Buffalo,  NY. 
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■AN  ERGONOMIC  APPROACH  TO- 
ERGONOMICS  DATA 


Engineering  Data  Compendium:  Human  Perception 
and  Performance  edited  by  Kenneth  R.  Boff  and 
Janet  E.  Lincoln  (1988) 


ngineering  Data  Compendium :  Human  Perception  and 
Performance  is  a  landmark  human  engineering  reference  for 
system  designers  who  need  an  easily  accessible  and  reliable  source  of 
human  performance  data.  Editors  Kenneth  R.  Boff  and  Janet  E.  Lincoln 
make  understanding,  interpreting,  and  applying  technical  information 
easy  through  their  innovative  format.  This  four  volume,  2758  page  set 
features  nearly  2000  figures,  tables,  and  illustrations  in  several  well- 
structured  approaches  for  accessing  information.  Brief  encyclopedia- 
type  entries  present  information  about  basic  human  performance  data, 
human  perceptual  phenomena,  models  and  quantitative  laws,  and 
principles  and  nonquantitative  laws.  Section  introductions  provide  an 
overview  of  topical  areas.  Background  information  and  tutorials  help 
users  understand  and  evaluate  the  material. 


For  further  information  on  the  Engineering  Data  Compendium ,  contact 
CSERIAC  at  (513)  225-4842. 
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Vs  automation  the  solution 
to  all  production  prob¬ 
lems?  It  seemed  for  a 
while  that  might  be  the  case.  But  as 
Dr.  Martin  Helander,  State  University 
of  New  York,  points  out  in  his 
feature  article  on  “Automation  and 
Human-Computer  Interaction  in 
Manufacturing,”  there  are  pitfalls  with 
automation.  Sometimes  manual 
methods  are  better  than  automated 
methods.  To  learn  more  about  the 
intricacies  of  automation  and  manu¬ 
facturing,  read  this  article  which 
Martin  based  on  his  highly  success¬ 
ful  presentation  as  part  of  the  1993 
Human  Engineering  Division, 
Armstrong  Laboratory  Colloquium  Se¬ 
ries:  The  Human-Computer  Interface. 

Another  speaker  from  that  series, 
Dr.  John  Stern  of  Washington  Univer¬ 
sity,  spoke  on  “The  Eyes:  Reflector 
of  Attentional  Processes.”  Dr.  June 
Skelly,  from  the  Crew  Systems  Integra¬ 
tion  Branch  in  our  division,  provides 
a  synopsis  of  his  presentation;  it  is 
accompanied  by  an  edited  transcript 


of  a  conversation  that  I  had  with 
John  during  his  visit. 

Last  spring,  the  Human  Engineering 
Division  of  the  Armstrong  Laboratory 
hosted  a  workshop  on  Electronic  Im¬ 
aging  of  the  Human  Body.  Jennifer 
Whitestone,  Design  Technology 
Branch,  Human  Engineering  Division, 
details  the  mission  of  the  workshop  in 
this  issue.  Twenty-three  invited  speak¬ 
ers  presented  papers  on  a  variety  of 
topics  related  to  this  area.  These  have 
been  collected  in  a  single  volume 
which  is  available  through  CSERIAC 
for  $35. 

Collecting  data  through  surveys, 
questionnaires,  and  interviews  can 
often  be  a  challenge.  However,  the 
staff  at  Bernier  &  Associates  have 
made  that  task  easier  through  the 
development  of  a  software  program, 
INFORUM.  Leo  Bernier  has  provided 
an  overview  of  INFORUM  and  some 
of  its  features.  For  example,  it  can 
collect  data  from  different  people  in 
different  locations  at  different  times! 
This  flexibility  in  data  collection  al¬ 
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lows  for  broad  participation,  without 
the  expense  and  aggravation.  And  this 
is  only  one  of  its  features! 

After  more  than  four  years  as  the 
Associate  Director  of  CSERIAC,  Dr. 
Larry  Howell  has  moved  on  to  other 
duties  at  the  University  of  Dayton 
Research  Institute.  While  Dr.  Howell 
was  at  CSERIAC,  the  organization  grew 
from  an  initial  four-person  operation 
to  a  staff  of  25;  during  this  same  period, 
the  revenues  doubled  every  year. 

Dr.  Howell  is  succeeded  by  Mr.  Don 
Dreesbach  who  has  led  CSERIACs 
technical  area  task  management  since 
January  1992.  Mr.  Dreesbach  has  a 
distinguished  Air  Force  career,  includ¬ 
ing  operational  flying  and  laboratory 
directorate  management,  as  well  as  a 
number  of  years  of  contract  technical 
management.  We  offer  our  best  wishes 
to  both  in  their  new  positions!  • 

Reuben  “Lew” Harm,  Ph.D.  is  the  Contract¬ 
ing  Officer's  Technical  Representative 
(COTR)  who  serves  as  the  Government 
Manager  for  the  CSERIAC  Program. 


BCPE  Certification  Requirements 
Revised 


The  Board  of  Certification  in  Professional 
Ergonomics  (BCPE)  has  revised  its  certification 
requirements  for  ergonomists  and  human 
factors  professionals.  Starting  January  1,  1994, 
certification  criteria  will  be  (1)  a  master’s 
degree  in  ergonomics/human  factors  or 
equivalent  educational  background;  (2)  four 
years  of  full-time  professional  practice  in 
ergonomics  with  an  emphasis  on  ergonomic 
design;  (3)  submission  of  a  work  product 
demonstrating  the  application  of  ergonomics  to 


a  product,  process,  or  environment;  and  (4)  a 
passing  score  on  a  written  certification 
examination  administered  by  the  BCPE. 

Previously,  however,  experienced  practitio¬ 
ners  were  certified  under  a  “waiver  of  examina¬ 
tion”  procedure,  for  which  the  minimum 
requirements  were  a  master’s  degree  in  ergo¬ 
nomics  or  a  closely  related  field  and  seven 
years  of  demonstrable  experience  in  the  prac¬ 
tice  of  ergonomics.  Applications  are  available 
for  $10  and  the  application  processing  fee 
is  $200  (to  which  the  $10  application  fee  may 
be  applied),  and  the  annual  renewal  fee  is  $73. 
For  further  information  or  an  application, 
please  contact: 

BCPE 

Office  of  the  Executive  Director 

P.O.  Box  2811 

Bellingham,  WA  98227-2811 

Telephone  (206)  671-7601,  fax  (206)  671-7681 


Human  Factors  and  Ergonomics  Society 
Placement  Service 


The  Human  Factors  and  Ergonomics  Society 
(HFES)  offers  an  opportunity  for  employers 
seeking  human  factors/ergonomics  expertise  to 
tap  into  the  tremendous  knowledge  base  and 
range  of  experience  of  HFES  members  and 
other  professionals  in  related  fields. 

Candidates  are  computer-matched  with  avail¬ 
able  jobs  on  the  following  criteria: 

Highest  degree  held 
Major  field  of  study 
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Years  of  experience 
Salary 

Areas  of  interest 
Employment  sector 
Geographical  Area 
Type  of  position 

The  HFES  Placement  Service  can  help  in 
filling  full-time,  part-time,  or  consulting  posi¬ 
tions.  Both  members  and  nonmembers  may  use 
the  service  for  renewable  four-month  terms. 
Anonymity  will  be  maintained  if  requested  by 
the  user. 

Call  HFES  at  (310)  394-1811  or  fax  (310)  394- 
2410  to  obtain  rates  and  application  forms. 


Calendar 


March  16-20, 1994 

San  Antonio,  TX,  USA 

EDRA  25,  the  25th  Annual  Meeting  of  the 
Environmental  Design  Research  Association. 
Contact  EDRA  Business  Office,  P.O.  Box  24083, 
Oklahoma  City,  OK  73214;  (405)  843-4863. 

May  8-12,  1994 

San  Antonio,  TX,  USA 

Aerospace  Medical  Association  65th  Annual 
Scientific  Meeting.  Contact  Pamela  Day,  Aero¬ 
space  Medical  Association,  320  S.  Henry  St., 
Alexandria,  VA  22314;  (703)  739-2240. 

June  12-17, 1994 

San  Jose,  CA,  USA 

SID  ’94,  Society  for  Information  Display  Interna¬ 
tional  Symposium,  Seminar,  and  Exhibition. 
Contact  Joyce  E.  Farrell,  SID  ’94  Conference 
Chair,  Hewlett-Packard  Labs,  P.O.  Box  10490, 
Palo  Alto,  CA  94303-0969;  (415)  857-2807,  fax 
(415)  857-4320. 

April  7-8,  1994 

Washington,  DC,  USA 

1st  Automation  Technology  and  Human  Perfor¬ 
mance  Conference.  Contact  Mustapha  Mouloua, 
Cognitive  Science  Labs,  Catholic  University  of 
America,  Washington,  DC  20064;  (202)  319- 
5825. 

May  21-25,  1994 

Washington,  DC,  USA 

Association  for  the  Advancement  of  Medical 
Instrumentation  29th  Annual  Meeting  and  Ex¬ 
position.  Contact  AAMI  Education  Department, 
2220  Washington  Blvd. ,  Suite  400,  Arlington,  VA 
22201-4598;  (703)  525-4890,  fax  (703)  276-0793. 

July  6-8, 1994 

Manchester,  UK 

4th  International  Conference  on  Human  As¬ 
pects  of  Advanced  Manufacturing  and  Hybrid 
Automation.  Contact  Paul  Kidd,  Cheshire 
Henbury  Research  and  Consultancy,  Tamworth 
House,  P.O.  Box  103,  Macclesfield,  SK11  8UW, 

UK;  (44)  625-619-313,  (44)  625-619-060. 

April  19-22, 1994 

University  of  Warwick,  UK 

Ergonomics  Society  Annual  Conference.  Con¬ 
tact  Conference  Manager,  Devonshire  House, 
Devonshire  Sq.,  Loughborough,  Leichestershire 
LE11  3DW,  UK;  (44)  509-234904. 

May  23-27, 1994 

Dayton,  OH,  USA 

NAECON  ’94,  National  Aerospace  and  Electron¬ 
ics  Conference.  Contact  NAECON  ’94,  P.O.  Box  , 
31341,  Dayton,  OH  45431-0341,  or  call  Thomas 

J.  Gaudian,  (513)  427-4267,  fax  (513)  427-4675. 

August  2-6,  1994 

St.  Petersburg,  Russia 

East-West  International  Conference  on  HCI. 
Contact  Claus  Unger,  Praktische  Informatik  II, 
Fernuniversitat,  Feithstr.  140,  D-58084  Hagen, 
Germany,  or  Email  ewinfo.chi@xerox.com. 

April  24-28,  1994 

Boston,  MA,  USA 

CHI  ’94:  Association  for  Computing  Machinery 
on  Human  Factors  in  Computing  Systems.  Con¬ 
tact  Thomas  Hewett,  Drexel  University,  Dept,  of 
Psychology/Sociology/Anthropology,  Central 
Receiving,  33rd  &  Ludlow  Sts.,  Philadelphia,  PA 
19104;  (215)  590-8616. 

June  7-10, 1994 

San  Antonio,  TX,  USA 

Industrial  Ergonomics  and  Safety  Conference. 
Contact  F.  Aghazadeh,  IMSE  Department,  Loui¬ 
siana  State  University,  Baton  Rouge,  LA  70803; 
(504)  388-5367,  fax  (504)  388-5990. 

August  15-19,  1994 

Toronto,  Canada 

12th  Triennial  Congress  of  the  International 
Ergonomics  Association.  Contact  IEA  ‘94  Secre¬ 
tariat,  c/oJPdL  Multi  Management  Inc.,  Toronto 
Dominion  Center,  55  King  St.  W,  #2550,  Toronto, 

ON,  Canada  M5KIE7;  (4l6)  784-9336,  fax  (4 1 6) 
784-0808. 

Notices  for  the  calendar  should  be  sent  at  least  four  months  in  advance  to: 


CSERIAC  Gateway  Calendar,  AL/CFH/CSERIAC  Bldg  248,  2255  H  Street,  Wright- Patterson  AFB  OH  45433-7022 


Human  Factors  and  Ergonomics  Society  Minneapolis,  MN. 

Fellows  Elected  Alvah  C.  Bittner,  Jr.,  Senior  Research  Scien¬ 

tist,  Battelle  Human  Affairs  Research  Center, 
Seattle,  WA. 

Deborah  A.  Boehm-Davis ,  Associate  Profes¬ 
sor,  Department  of  Psychology,  George  Mason 
University,  Fairfax,  VA. 

Three  distinguished  nominees  were  recently  Fellow  status  requires  a  minimum  of  five 

elected  Fellows  of  the  Human  Factors  and  years  of  membership,  ten  years  of  professional 

Ergonomics  Society.  Joining  the  Society’s  other  experience  in  the  field,  outstanding  and  demon- 

216  Fellows  are:  strable  contributions  to  human  factors,  three 

years  of  direction  or  supervision  of  significant 
Philip  L .  Ackerman ,  Professor,  Department  human  factors  efforts,  and  at  least  one  year’s 

of  Psychology,  University  of  Minnesota,  service  to  the  Society. 
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Human  Engineering  Division,  Armstrong  Laboratory  Colloquium  Series 

The  Eyes:  Reflector  of  Attentional  Processes 

John  A.  Stern 
Synopsis  by  June  J.  Skelly 


Editor's  Note:  Following  is  a  synopsis  of  a 
presentation  by  Dr.  John  A.  Stern,  as  the 
third  speaker  in  the  1993  Human  Engi¬ 
neering  Division ,  Armstrong  Laboratory 
Colloquium  Series:  The  Human-Computer 
Interface.  This  synopsis  was  prepared  by 
Dr.  June  J.  Skelly,  Research  Psychologist 
with  the  Crew  Systems  Integration  Branch, 
Human  Engineering  Division,  Armstrong 
Laboratory.  JAL 

Or.  John  Stern  introduced 
his  discussion  topic  by 
posing  the  question:  Why 
should  we,  as  human  factors  research¬ 
ers,  be  interested  in  assessing  the  gaze 
control  system?  While  it  is  acknowl¬ 
edged  that  the  gaze  control  system 
functions  to  acquire  information,  it  is 
unlikely  that  few  of  us  have  enter¬ 
tained  the  idea  that  this  system  could 
also  provide  information  about  how 
we  process  this  information.  Dr.  Stern 
maintains  that  the  gaze  control  system 
(eyes,  head,  and  body)  may  in  fact,  be 
an  important  source  for  providing  valu- 
able  information  regarding  an 
operator’s  level  of  attention,  alertness, 
and  fatigue. 

In  our  high-tech  environments,  op¬ 
erators  spend  more  time  monitoring 
rather  than  operating  equipment.  With 
current  technology,  the  likelihood  of 
equipment  malfunction  is  small.  With 
infrequent  need  on  the  part  of  opera¬ 
tors  to  intervene,  their  task  has  be¬ 
come  a  vigilance  task.  Vigilance  tasks 
are,  by  definition,  tasks  requiring  sus¬ 
tained  attention  but  with  little  need  to 
perform  specific  operator  functions. 
The  prevailing  assumption  has  been 
that  the  operator  will  adjust  to  changes 
in  task  requirements  when  necessary, 
continuing  to  perform  efficiently.  Ac¬ 
cording  to  Dr.  Stern,  that  assumption 
now  has  to  be  challenged.  He  noted 
that  it  has  been  well  documented  that 
the  human  operator  does  not  perform 
such  monitoring  functions  reliably, 


and  that  time-on-task  (TOT)  is  a  major 
component  of  performance  decrements. 
Human  error  appears  more  and  more 
frequently  to  be  a  major  contributor  to 
fatal  accidents  and  near  misses  in  the 
aviation  environment.  What  can  be 
done  to  improve  this  picture? 

Dr.  Stern  suggests  that  a  fruitful 
approach  to  investigating  questions  of 
performance  decrements  in  long  dura¬ 
tion  tasks,  would  be  to  monitor  the 
operator  for  alterations  in  attention 
to  the  assigned  task.  He  argues  that 
one  could  then  take  corrective  action 
of  one  kind  or  another  when  vigilance 
wanes,  hopefully  before  important 
events  are  missed.  This  approach 
requires  that  we  conceive  of  the  op¬ 
erator  as  an  important  component  of 
the  system.  Dr.  Stern  concluded  his 
introduction  by  proposing  that  inves¬ 
tigations  of  our  gaze  control  system 
should  be  an  important  part  of 
“system  evaluations.” 

During  the  remainder  of  his  presen¬ 
tation,  Dr.  Stern  discussed  evaluating 
gaze  control  system  components,  and 
how  we  might  use  this  information  to 
ascertain  an  operator’s  level  of  atten¬ 
tion  and  fatigue  He  limited  his  discus¬ 
sion  topics  to  head  and  eye  movement 
research,  noting  that  investigations  of 
the  body  (e.g.,  orientation  and  loco¬ 
motion)  have  not  been  the  focus  of  his 
own  research. 

Gaze  Control  and  Attention  and 
Fatigue  Processes 

Head  Movements 

By  way  of  introducing  this  research, 
Dr.  Stern  began  by  asking  the  audi¬ 
ence  to  think  about  the  meaning  of  the 
word  “democracy.”  He  asked  the  group 
whether  they  had  noticed  making  ei¬ 
ther  a  head  movement  or  shifting  their 
gaze  while  pondering  the  meaning  of 
“democracy;”  most  did  not.  Yet,  Dr. 
Stern  reported  that  he  had  observed 


this  behavior  in  our  group.  In  fact, 
naturalistic  observations  have  shown 
that  people  typically  move  their  head 
or  their  eyes  when  thinking.  For 
example,  observations  of  young  inex¬ 
perienced  readers  show  they  are  more 
likely  to  make  head  movements  than 
adults.  This  is  also  the  case  in  experi¬ 
mental  situations;  adults  will  typically 
make  head  movements  when  reading 
aloud,  but  not  when  reading  silently. 
Head  movements  are  also  likely  to 
occur  when  reading  difficult  text  as 
compared  to  easy  text. 

The  timing  of  head  movements  with 
respect  to  the  eye  also  appears  to 
reveal  aspects  of  information  predict¬ 
ability.  For  example,  in  laboratory 
situations,  Dr.  Stern  and  his  colleagues 
have  shown  that  in  a  search  task  when 
stimulus  position  is  cued,  the  head  will 
move,  on  average,  100  ms  ahead  of  an 
eye  movement  to  the  periphery.  How¬ 
ever,  in  an  uncued  situation,  the  head 
lags  30-60  ms  behind,  a  saccadic  eye 
movement  when  acquiring  a  periph¬ 
eral  target.  Thus,  head  movements 
made  to  acquire  information  also  tell 
us  a  great  deal  about  the  process  itself. 

Eye  Movements 

By  far,  eye  movements  are  Dr.  Stern’s 
favorite  research  topic  and  by  way  of 
introducing  this  aspect  of  the  gaze 
control  system,  he  gave  an  example  of 
how  we  can  make  some  inferences 
about  an  individual’s  information  pro¬ 
cessing  style  by  naturalistic  observa¬ 
tion.  For  example,  if  asked  to  spell  the 
word  mother  backwards,  some  indi¬ 
viduals  will  make  a  lateral  eye  move¬ 
ment  to  the  left  and  others  will  shift 
their  gaze  to  the  right.  Those  individu¬ 
als  that  shift  gaze  to  the  left  appear  to 
be  able  to  visualize  the  word  mother 
and  then  simply  read  off  the  letters  R- 
E-H-T-O-M,  while  those  that  shift  their 
gaze  to  the  right  appear  to  rely  on 
Continued  on  page  8 
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auditory  or  other  symbolic  strategies, 
such  as  repeating  the  word  “mother, 
mother,  mother...”  until  spelling  is 
complete.  Lateral  eye  movements  are 
but  one  aspect  of  eye  movements  that 
have  been  shown  to  be  affected  by 
cognitive  and  TOT  processes. 

Dr.  Stern’s  pioneering  research  in¬ 
vestigating  the  eye  blink  has  shown 
eye  blinks  do  not  occur  at  random. 
Blink  frequency,  latency,  and  duration 
have  been  shown  to  be  affected  by 
cognitive  demands  and  TOT.  We 
blink  when  we  can  take  “time  out” 
from  processing.  For  example,  when 
required  to  commit  information  to 
memory,  blink  latency  increases  as  the 
number  of  items  to  be  committed  to 
memory  increases.  Pupillary  dilation 
is  also  significantly  related  to  the  num¬ 
ber  of  items  in  a  memory  set  as  well; 
the  higher  the  cognitive  demands,  the 
greater  the  increase  in  pupillary  dila¬ 
tion.  Blink  frequency  and  blink  dura¬ 
tion  also  reflect  attentional  demands  in 
an  operational  situation.  For  example, 
B-52  pilots  in  a  simulated  long-dura¬ 
tion  mission  blinked  significantly  less 
than  their  copilots.  However,  when 
pilots  and  copilots  reversed  roles,  their 
pattern  of  eye  blinks  reversed  as  well. 
That  is,  pilots  assuming  a  copilot  role 
blinked  more  frequently  than  copilots 
that  now  assumed  pilot  responsibili¬ 
ties.  Eye  blink  duration  was  also 
affected  by  high  attentional  demands. 
In  this  same  study,  individuals  assum¬ 
ing  pilot  responsibilities  exhibited 
shorter  duration  eye  blinks  than  those 
in  the  copilot  role.  These  changes 
were  attributable  to  the  task  demands 
and  not  to  the  individual. 

Dr.  Stern’s  research  has  also  shown 
that  eye  blink  measures  can  provide 
valuable  information  about  TOT  ef¬ 
fects.  In  vigilance  situations,  most 
tasks  of  concern  involve  visual  stimuli. 
Therefore,  if  the  eyes  are  closed  while 
we  are  supposed  to  be  taking  in  infor¬ 
mation,  such  information  is  missed. 
Further,  if  the  eyes  are  open  and 
immobile  for  long  periods  of  time 
when  scanning  is  required,  once  again 
we  would  suspect  that  vigilance  is 
compromised.  There  is  in  fact,  a  large 
literature  of  experimental  results  dem¬ 
onstrating  that  blink  frequency  in¬ 
creases  as  a  function  of  TOT.  One 
example  of  TOT  effects  is  the  appear¬ 
ance  of  blink  flurries,  a  series  of  eye 


blinks  occurring  in  rapid  succession. 
Blink  flurries  often  occur  after  the 
eyes  have  been  immobile  for  a  period 
of  time.  It  is  thus  not  outside  the  realm 
of  possibility  that  the  increase  in  blink 
frequency  as  a  function  of  TOT  may 
be  associated  with  “attentional  drop¬ 
outs”  and  hence,  lapses  in  vigilance. 
Dr.  Stern  suggests  that  analysis  of 
specific  head  and  eye  movement  rela¬ 
tionships  can  provide  valuable  infor¬ 
mation  regarding  an  operator’s  ability 
to  maintain  a  vigilant  state. 


Gaze  Control  and  a  Simulated 
Air  Traffic  Control  Study 

During  the  final  segment  of  his  pre¬ 
sentation,  Dr.  Stern  presented  data 
from  a  recent  study  designed  to  exam¬ 
ine  changes  in  the  gaze  control  of 
operators  as  a  function  of  TOT. 

The  data  were  obtained  during 
a  simulated  air  traffic  control  task. 
Subjects  performed  the  task  in  two- 
hours  sessions  each,  on  three 
separate  occasions,  for  a  total  of  six 


Figure  1(a).  A  flurry  of  blinks  in  response  to  a  critical  stimulus  event  indicating 
a  re-alerting  of  the  attentional  system . 
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Figure  1(b).  An  example  of  a  time-locked  gaze  control  component  associated 
with  efficient  visual  processing. 
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Figure  2.  A  pattern  of  eye  and  head  movements  associated  with  a  low 
vigilant  state. 


hours.  The  task  involved  monitoring 
a  CRT  where  two  vectors  oriented 
from  the  lower  right  to  upper  left 
each  carried  eight  aircraft  to  be  moni¬ 
tored.  During  each  of  the  two-hour 
test  periods  there  were  44  “unexpected” 
events  that  occurred  requiring 
operator  action.  These  events  were  of 
three  types:  (1)  two  aircraft  flying 
at  the  same  altitude,  (2)  aircraft  flying 
outside  of  a  vector,  and  (3)  loss 
of  altitude  information  presented 
for  an  aircraft. 

Head  movements  were  recorded 
using  a  photoelectric  device.  Eye  move¬ 
ments  in  the  horizontal  and  vertical 
planes  were  recorded  using  electro- 
oculographic  measures  (the  EOG  is 
not  a  measure  of  muscle  activity  but 
reflects,  it  is  believed,  changes  in  the 
potential  difference  between  the  retina 
and  cornea  as  the  latter  shifts  in  two- 
dimensional  space).  Blink  measures 
abstracted  during  operator  perfor¬ 
mance  included  blink  rate  as  a  func¬ 
tion  of  TOT,  blink  amplitude,  blink 
duration,  and  saccadic  eye  movements 
measured  per  unit  time  to  determine 
amount  of  search  being  done  by  the 
operator.  It  was  expected  that  saccade 
frequency  would  decrease  over  time. 
In  addition  to  these  measures,  the 
timing  relationships  between  blinks 
and  saccades  were  measured.  In  Dr. 
Stern’s  opinion,  an  alert  system  will 
show  blinks  and  saccades  to  be  tightly 
time-locked,  since  there  can  be  no 
visual  intake  during  either  a  blink  or  a 
saccade.  An  example  of  such  time 
locking  is  shown  in  Figure  1,  panels 
(a)  and  (b).  In  panel  (a)  we  see  the 
subject  has  been  alerted  to  a  critical 
stimulus  event  that  requires  a  complex 
decision  and  appropriate  input  re¬ 
sponse.  Note  the  flurry  of  blinks  oc¬ 
curring  near  the  end  of  the  stimulus 
event.  Figure  1  (b)  shows  a  continu¬ 
ation  of  this  record.  Here,  just  prior  to 
response  input,  we  see  an  example  of 
head  movement,  eye  blinks,  and  sac¬ 
cadic  time-locking.  Notice  that  as  the 
head  shifts  to  the  right  there  are  con¬ 
comitant  blinks  and  saccadic  activity 
to  the  right.  Then,  immediately  before 
response  input,  the  head  readjusts, 
returns  to  position,  and  the  eye  makes 
a  compensatory  saccade.  During  this 
time  no  visual  intake  can  occur.  Thus, 
this  tight  time-locking  of  head  and 
eye  movements  is  an  efficient  way  for 


the  system  to  reduce  visual  intake 
“down  time,” 

However,  as  an  operator  begins  to 
tire  (prior  to  any  noticeable  perfor¬ 
mance  decrement),  Dr.  Stern  reasons 
that  one  should  see  an  emerging  pat¬ 
tern  of  decoupling  between  blinks 
and  saccades.  Figure  2  shows  an  ex¬ 
ample  of  an  “attentional  dropout”  pe¬ 
riod  as  a  function  of  TOT. 

In  this  record  we  see  the  subject’s 
head  begin  to  wobble,  followed  by 
long  closure  duration  blinks  and  then 
full  eyelid  closures.  The  horizontal 
EOG  channel  indicates  that  an  effi¬ 
cient  saccadic  scan  pattern  (see  Fig.  1) 
has  been  replaced  by  the  eyes  making 


large  excursions  in  both  directions.  It 
is  highly  unlikely  that  this  subject 
would  have  picked  up  a  critical  stimu¬ 
lus  event  had  one  occurred  during  this 
time  frame. 

Experimental  Results 

Results  of  this  simulated  air  traffic 
control  study  showed  that  blink  mea¬ 
sures  did  indicate  a  number  of  signifi¬ 
cant  TOT  effects.  Blink  rate  showed  a 
significant  increase  over  time,  as  was 
also  true  for  the  occurrence  of  blink 
flurries  (defined  as  a  minimum  of 
three  blinks  occurring  in  a  three  sec- 
Continned  on  page  10 


Table  1. 

Significant  Changes  in  Blink  Measures  as  a  Function 
of  Time-on-Task  (TOT). 


Measure 

Significance  Level 

Blink  Rate  (increase) 

p  <.0008 

Frequency  of  Blink  Flurries  (increase) 

p  <.01 

Blink  Duration  (increase) 

p  <.03 

Saccade  Rate  (decrease) 

p  <.0004 

Fixation  Duration  (increase) 

p  <.04 

Saccade  Amplitude  (decrease) 

p  <.0000 
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ond  period).  As  predicted,  saccade 
rate  decreased  as  a  function  of  TOT. 
Long  duration  fixation  pauses  (in  ex¬ 
cess  of  2  seconds)  increased  over  time 
suggesting  that  as  fixation  duration 
increased  there  was  also  less  scanning 
by  the  subject  as  indicated  by  the 
decrease  in  saccade  rate.  These  data 
are  summarized  in  Table  1. 

Summarizing,  Dr.  Stern  noted  that 
these  are  but  a  few  examples  demon¬ 
strating  how  information  from  biologi¬ 
cal  measures  might  be  used  to  make 
inferences  about  the  operator’s  “state” 
or  ability  to  respond  to  infrequently 
occurring  events.  These  aspects  of  the 
gaze  control  system  can  be  especially 
useful  for  the  evaluation  of  equipment 
operators,  especially  those  individual 
who  utilize  visual  input  as  a  major 
source  of  information  on  which  action 
must  be  taken.  # 

John  A.  Stem,  Ph.D.,  is  Chairman  of  the 
Department  of  Psychology,  Washington 
University,  St.  Louis,  MO. 

References 

Fogarty,  C.,  &  Stern,  J.A.  (1989).  Eye 
movements  and  blinks:  Their  relation¬ 
ship  to  higher  cognitive  processes. 
International  Journal  of  Psychophysi¬ 
ology,  8,  35-42. 

Goldstein,  R.,  Bauer,  L.O.,  &  Stern,  J. 
A.  (1992).  Effect  of  task  difficulty  and 
interstimulus  interval  on  blinking  pa¬ 
rameters.  International  Journal  of 
Psychophysiology,  13,  111-117. 

Stern,  J.A.,  Boyer,  D.,  Schroeder,  D.J., 
Touchstone,  R.M.,  &  Stoliarov,  N.  (In 
press).  Blinks,  saccades,  and  fixation 
pauses  during  vigilance  performance. 
Federal  Aviation  Administration, 
Office  of  Aviation  Medicine  Report. 

Stern,  J.A.,  &  Dunham,  D.N.  (1990). 
The  ocular  response  system.  In  J.T. 
Caccioppo  &  L.L.G.  Tassinary  (Eds.), 
Principles  of  psychophysiology:  Physi¬ 
cal,  social,  and  inferential  elements. 
Cambridge,  MA:  Cambridge  Univer¬ 
sity  Press. 

Stern,  J.A.,  &  Skelly,  J.J.  (1984).  The 
eye  blink  and  workload  considerations. 
Proceedings  of  the  28th  Annual  Meet¬ 
ing  of  the  Human  Factors  Society,  2, 
942-944. 


Scenes  from  the  Human  Engineering  Division, 
Armstrong  Laboratory  Colloquium  Series: 


Dr.  Stem  stressed  the  point  that  the  gaze  control  system  provides  vital  information  about 
one’s  level  of  attention. 


Following  his  presentation,  Dr.  Stem  spoke  with 
Engineering  Division  of  Armstrong  Laboratory. 


veral  vision  scientists  from  the  Human 
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Human  Engineering  Division,  Armstrong  Laboratory  Colloquium  Series 

A  Conversation  with  John  Stern 

Reuben  L.  Hann 


Editor’s  Note:  The  following  is  an  edited 
transcript  of  a  conversation  with  Dr.  John 
A.  Stern,  Washington  University,  ivho  had 
just  made  a  presentation  as  the  third  speaker 
in  the  1993  Human  Engineering  Division, 
Armstrong  Laboratoty  Colloquium  Series: 
The  Human-Computer  Interface.  The 
interviewer  was  Dr.  Lew  Hann,  CSERIAC 
COTR.  JAL 

OSERIAC:  You  used  to  be 
involved  in  studies  using 
brain  wave  recording 
[EEG]  techniques;  how  did  you  get 
interested  in  the  area  of  eye  blink 
measures? 

Dr.  Stern:  I  was  doing  some  EEG 
studies  for  the  Army  aviation  people. 
They  were  concerned  about  the  pos¬ 
sibility  of  inducing  seizures,  due  to 
the  stroboscopic  effect  of  helicopter 
rotor  blades  breaking  up  sunlight.  We 
demonstrated  to  them  that  it  would 
not  be  a  problem;  but  in  the  process 
of  collecting  in-flight  data,  an  aviation 
psychologist  colleague  and  I  noticed 
that  the  pilots,  who  were  in  transition/ 
training  to  fly 
larger  helicopters, 
hardly  ever  moved 
their  heads.  We  de¬ 
cided  to  find  out 
what  they  were 
doing  with  their 
eyes  during  flight 
activity.  We  devel¬ 
oped  eye  move¬ 
ment  amplifiers 
and  started  recording  in  the  helicop¬ 
ter.  That’s  what  got  me  started  look¬ 
ing  at  what  people  do  with  their  eyes 
as  they  process  information. 

CSERIAC:  Why  the  eyeblink  in 
particular? 

Dr.  Stern:  Well,  as  we  did  our 
average  evoked-response  studies,  we 
found  people  frequently  blinked  at 
the  point  in  time  when  we  were  most 
interested  in  extracting  information, 


which  contaminated  the  results.  The 
eyeblink  signals  were  considered 
noise,  or  artifact.  So  I  thought,  instead 
of  telling  people  not  to  blink,  why  not 
study  the  blink  itself — since  it  seemed 
to  occur  during  points  in  time  associ¬ 
ated  with  the  processing  of  informa¬ 
tion.  In  other  words,  we  used  “noise” 
as  signal.  That’s  how  we  got  inter¬ 
ested  in  the  timing  of  eye  blinks. 

CSERIAC:  A  colleague  said  that 
you  sometimes  refer  to  the  eyeblink 
as  “the  brain’s  exclamation  point”. 
Could  you  explain  that? 

Dr.  Stern:  It’s  an  exclamation  point 
in  terms  of — here’s  a  point  in  time 
where  you  can  take  a  brief  rest,  where 
you  can  take  time  out  from  taking  in 
or  processing  information.  In  that 
sense,  it’s  an  exclamation  mark.  It 
indexes  a  specific  event  with  respect 
to  the  processing  of  information. 

CSERIAC:  During  your  presenta¬ 
tion  you  referenced  the  earlier  pupil¬ 
lary  response  work  of  Beatty.  I  re¬ 
member  that  years  ago  there  was  a 


flurry  of  research  examining  the  pu¬ 
pillary  response  in  a  variety  of  set¬ 
tings;  the  result  was  that  it  ended  up 
being  correlated  with  a  whole  variety 
of  behavioral  processes.  There  was 
criticism  by  someone  at  the  time  that 
it  correlated  with  “just  about  every¬ 
thing,”  and  was  therefore  of  question¬ 
able  value  in  inferring  internal  psy¬ 
chological  events.  Apparently  it  has 
found  new  respect  since  then. 

Dr.  Stern:  The  earliest  work  using 


pupil  dilation  was  by  Eckhardt  Hess, 
who  used  it  to  study  affect.  It  was  his 
conviction  that  pupillary  dilation  was 
associated  with  positive  affect,  con¬ 
striction  with  negative.  The  problem 
was  that  he  used  pictures  as  the  stimuli, 
with  no  control  for  the  amount  of  light 
impinging  on  the  eye.  Beatty’s  work 
was  important  because  he  removed 
that  potential  confound  by  presenting 
the  stimuli  auditorily.  The  pupillary 
response  then  is  a  reflector  of  what  is 
going  on  inside,  rather  than  a  reflector 
of  what  is  coming  in. 

CSERIAC:  When  you  worked  for 
the  CIA,  were  you  trying  to  use 
eyeblink  as  a  kind  of  “lie  detector”? 

Dr.  Stern:  Yes,  we’ve  done  some 
work  looking  at  what  happens  when 
people  are  required  to  tell  a  lie,  and  to 
see  if  can  we  use  information  from  the 
eyes  to  infer  about  whether  they  are 
answering  truthfully  or  deceptively. 
As  you  may  know,  the  common  poly¬ 
graph  technique  assumes  that  lying  is 
associated  with  affective  arousal,  and 
the  polygraph  gives  some  sort  of  in¬ 
dex  of  the  arousal 
associated  with  ly¬ 
ing.  We  reasoned 
that  if  you  are  re¬ 
quired  to  lie,  you 
are  required  to 
more  thoroughly 
process  informa¬ 
tion  cognitively.  So 
what  we  looked  at 
was  the  nature  of 
eye  movements  as  people  were  ac¬ 
quiring  information.  They  read  a  ques¬ 
tion  on  a  CRT.  We  could  identify  when 
they  had  finished  reading  the  question, 
and  we  looked  at  the  eye  movements 
between  that  point  and  the  onset  of 
answering  the  question.  It  discrimi¬ 
nated  nicely  between  answering  de¬ 
ceptively  and  answering  truthfully. 

CSERIAC:  You  mentioned  the  phe¬ 
nomenon  of  lateral  eye  movement 
Continued  on  page  12 


“It  may  be  better  to  build  redundancy  into  the  equipment, 
since  humans  do  such  a  poor  job  at  maintaining  vigilance; 
putting  two  computers  in  tandem  may  be  more  reasonable 
than  having  a  human  in  the  system.” 
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when  persons  were  involved  in  cog¬ 
nitive  activity  —  that  some  persons 
looked  left,  others  right. 

Dr.  Stern:  This  lateral  eye  move¬ 
ment  phenomenon  is  interesting. 
There  was  some  research  in  this  area 
in  the  1970s,  and  then  it  disappeared. 
There  were  attempts  to  relate  lateral 
eye  movement  to  whether  you  were 
processing  with  the  right  or  left  hemi¬ 
sphere.  It  turned  out  to  be  nonsense. 
But  the  fact  is,  if  you  ask  100  people 
a  series  of  20  to  30  questions,  while 
watching  their  eye  movements  associ¬ 
ated  with  taking  in  the  question  and 
before  responding,  about  one-third 
will  move  their  eyes  to  the  left,  one- 
third  will  be  right-movers,  and  one- 
third  will  be  indeterminate.  So  it’s  not 
a  normal  distribution,  it’s  rectilinear. 

CSERJAC:  What  do  these  eye  move¬ 
ments  tell  you  about  the  person? 

Dr.  Stern:  Persons  who  are  left- 
movers  are  more  likely  to  form  a 
mental  picture  as  they  read  text;  they 
do  not  “see”  the  words,  but  rather  a 
vivid  mental  image  is  formed.  These 
persons  are  more  likely  to  prefer  highly 
descriptive  novels,  by  the  way.  But 
the  question  is,  why  does  one  person 
develop  one  strategy,  and  another 
person  develops  a  different  one?  And 
how  do  these  strategies  affect  our 
ability  to  learn?  For  instance,  I  knew  a 
student  who  had  severe  “math  anxi¬ 
ety.”  This  person  had  a  very  difficult 
time  doing  mental  arithmetic.  It  turned 
out  that  she  was  a  left-mover  who 
visualized  everything.  This  is  not  the 
optimum  strategy  for  such  cognitive 
tasks.  When  she  was  taught  some 
tricks  which  did  not  involve  visualiza¬ 
tion,  she  found  the  task  much  easier. 
Apparently  her  teachers  were  visualiz¬ 
es  and  taught  that  strategy  to  kids 
learning  arithmetic.  If  she’d  had  a 
teacher  who  was  aware  of  the  fact  that 
some  kids  don’t  learn  well  with  this 
strategy,  she  might  not  have  devel¬ 
oped  all  these  anxieties.  So  it  is 
important  to  know  that  not  everyone 
solves  problems  the  same  way, 

CSERIAC:  If  the  population  is 
really  composed  of  two  kinds  of  “pro¬ 
cessors,”  what  implications  might 
that  have  for  the  design  of  human- 
machine  interfaces? 

Dr.  Stem:  Not  only  the  design,  but 


also  the  selection  of  people  to  operate 
the  equipment-there  are  two  ways  to 
look  at  it.  Let  me  tell  you  an  anec¬ 
dote.  We  were  doing  some  work  in 
a  flight  simulator,  looking  at  pilots 
flying  with  information  presented  on 
the  helmet  visor.  We  were  observing 
their  head  movements  as  they  oper¬ 
ated,  and  what  was  interesting  to  me 
was  the  frequency  with  which  they 
looked  down  at  a  conventional  paper 
map  in  a  holder  attached  to  their 
knee.  Some  looked  at  the  map,  and 
apparently  formed  an  internal  repre¬ 
sentation,  because  they  did  not 
have  to  look  again  for  minutes.  Others 
had  to  continually  refer  back  to  the 
map,  because  they  could  not 
retain  the  image.  So  there  are  marked 
individual  differences  with  respect  to 
these  abilities. 

CSERIAC:  Should  there  be  a 

screening  process  to  differentiate 
individuals,  based  on  this  kind 
of  skill? 

Dr.  Stern:  Well,  the  question  I  have 
is,  can  people  be  trained  to  be  more 
attentive  to  these  visual  cues?  Can 
persons  who  are  poor  visualizers  be 
trained?  If  they  can,  then  selection 
would  not  be  necessary. 

CSERIAC:  One  of  CSERIAC’s  mis¬ 
sions  is  to  expedite  the  transfer  of 
information  from  the  research  labora¬ 
tory  to  the  field.  Do  you  have  any 
thoughts  about  this  process? 

Dr.  Stern:  Well,  the  way  scientists 
retrieve  information  from  the  literature 
is,  I’m  sure,  also  applied  by  the  engi¬ 
neering  world.  But  they  may  be  look¬ 
ing  at  the  wrong  literature.  It  seems 
that  the  engineers  of  today  should  be 
concerned  with  issues  of  the  effect  of 
humans  being  out  of  the  loop  on  their 
ability  to  deal  with  problems  which 
occur  with  the  in-the-loop  equipment. 
They  ought  to  become  concerned  with 
issues  of  the  effect  of  our  new  technol¬ 
ogy  on  the  human’s  ability  to  deal  with 
that  technology. 

CSERIAC:  One  of  our  related  hu¬ 
man  factors  organizations  has  an  inter¬ 
esting  motto:  “There  are  no  un¬ 
manned  systems.” 

Dr.  Stem:  Humans  have  to  be 
there-at  least  to  monitor.  Things  go 
wrong  frequently  enough  for  them  to 


have  to  intervene.  But  the  fact  is, 
computer-controlled  equipment  is 
becoming  increasingly  reliable,  so  the 
likelihood  that  a  person  will  be  called 
upon  is  decreasing.  It  may  well  be  that 
10  years  from  now  airplanes  will  fly 
without  pilots.  It  may  be  better  to 
build  redundancy  into  the  equipment, 
since  humans  do  such  a  poor  job  at 
maintaining  vigilance;  putting  two 
computers  in  tandem  may  be  more 
reasonable  than  having  a  human 
in  the  system.  To  many,  that’s  heresy, 
but  I  can  see  that  happening.  Consider 
the  issue  of  humans  in  space,  and 
what  has  been  contributed  by  putting 
humans-in-space,  rather  than  send¬ 
ing  equipment  up  to  perform  the 
functions.  Much  of  the  humans-in- 
space  effort  has  been  because  the 
Russians  have  done  it — because  it  has 
popular  appeal,  and  helps  to  sell  the 
space  program.  If  you  simply  send 
equipment  up,  it  doesn’t  have  that 
appeal.  In  the  future,  humans  will  not 
only  be  out  of  the  loop,  they  won’t 
even  be  there. 

CSERIAC:  Surely  there  will  be  a 
human  in  the  loop  somewhere? 

Dr.  Stern:  Oh  yes.  But  you  can  take 
over  control  from  the  ground  as  well 
as  in  the  air. 

CSERIAC:  There  is  the  problem  of 
trusting  the  hardware,  though. 

Dr.  Stern:  Right.  But  think  about 
what  pilots  are  permitted  to  do  on 
trans-Atlantic  flights.  They  are  now 
allowed  to  take  naps,  as  long  as  one 
crew  member  is  awake.  I’m  not  sure 
how  alert  that  one  is,  who  is  awake. 
There  are  frightening  examples.  Look 
at  the  frequency  with  which  tankers 
collide  on  the  high  sea  in  clear 
weather.  They  have  radar  systems 
available;  the  human  is  looking  at 
the  display,  but  doesn’t  see  the  other 
ship.  It  seems  that  the  technology  is 
available  where,  if  another  object  is 
moving  out  there,  you  can  alert 
personnel,  without  requiring  some¬ 
one  to  see  it.  You  can  simply  inform 
the  crew  that,  for  example,  if  no  action 
is  taken,  there  will  be  a  collision  in  15 
minutes.  The  same  thing  is  true  for 
aircraft,  but  the  time  scale  is  consider¬ 
ably  compressed,  of  course.  If  your 
eyes  close  for  three  seconds,  and  you 
are  closing  with  someone,  you  have  a 
major  problem.  # 
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Proceedings  of  the  Working  Group  on 
Electronic  Imaging  of  the  Human  Body 

Jennifer  Whitestone 


Ohe  Human  Engineering 
Division,  Armstrong  Labo¬ 
ratory,  in  cooperation  with 
the  Mallinckrodt  Institute  of  Radiol¬ 
ogy;  the  Washington  University  School 
of  Medicine;  and  the  Lister-Hill  Na¬ 
tional  Center  for  Biomedical  Commu¬ 
nication,  National  Library  of  Medicine, 
hosted  the  first  annual  meeting  of  the 
Cooperative  Working  Group  on  Elec¬ 
tronic  Imaging  of  the  Human  Body. 
The  working  group,  under  the  aus¬ 
pices  of  CSERIAC,  brought  together  31 
researchers  from  government,  indus¬ 
try,  and  academia  working  in  the  area 
of  medical  imaging  and,  more  gener¬ 
ally,  image  science,  to  discuss  issues 
associated  with  collecting,  archiving, 
transferring,  visualizing,  and  analyz¬ 
ing  image  data.  Although  applications 
for  three-dimensional  (3-D)  human 
body  image  data  range  from  recon¬ 
structive  surgery  to  designing  protec¬ 
tive  equipment,  the  challenges  faced 
by  medical  researchers,  anthro- 
pometrists,  educators,  prosthodontists, 
and  design  engineers  are  the  same: 
How  can  the  images  be  captured 
quickly  and  accurately?  Can  the  data 
be  reduced  to  a  manageable  size  for 
archiving,  manipulating,  and  visualiz¬ 
ing?  How  can  the  images  be  repre¬ 
sented  in  an  efficient,  yet  lossless  for¬ 
mat  to  allow  for  meaningful  and  timely 
analyses?  What  software  tools  and 
filters  are  available  for  handling  the 
data?  What  databases  of  human  body 
images  currently  exist  and  how  can 
these  images  be  made  accessible  to 
other  researchers  and  physicians?  What 
standards  exist  for  image  data  and 
what  methods  or  filters  are  available 
for  implementing  these  standards?  This 
working  group  has  encouraged  effec¬ 
tive  use  of  the  resources  from  all  the 
various  disciplines  involved  in  elec¬ 
tronic  imaging  of  the  human  body, 
identified  immediate  challenges  asso¬ 
ciated  with  handling  image  data,  and 
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developed  strategies  for  applications 
relevant  to  engineering,  medicine, 
entertainment,  and  education.  To 
address  image  data  compatibility  across 
numerous  platforms  and  with  applica¬ 
tions  software,  this  multi-disciplinary 
group  will  continue  to  provide  direc¬ 
tion  for  data  format  standards  that  will 
impact  future  industry  standards. 

Five  topic  areas  were  identified  to 
assist  in  categorizing  the  position  pa¬ 
pers  offered  by  each  of  the  working 
group  members.  These  areas  include 
(1)  development  of  surface  scanning 
systems,  (2)  data  storage  and  inter¬ 
change  format  standards,  (3)  calibra¬ 
tion,  validation,  and  evaluation  of  scan¬ 
ning  systems,  (4)  data  analysis,  image 
processing,  and  display,  and  (5)  physi¬ 
cally  based  modeling  of  deformable 
objects.  The  Proceedings ,  available 
through  CSERIAC  for  $35,  offer  a  brief 
overview  of  the  progress  and  issues 
associated  with  each  of  these  topic 
areas  and  provide  a  compilation  of  the 
position  papers  submitted  by  each  of 
the  23  speakers. 

Computer  networks  have  revolu¬ 


tionized  exchange  and  distribution  of 
visualization  technology  in  the  form  of 
“collaboratories.”  Collaboratories,  a 
combination  of  the  term  “collabora¬ 
tion”  and  “laboratories,”  are  an  elec¬ 
tronic  extension  of  independent  labo¬ 
ratories  containing  hardware,  software, 
people,  and  databases,  but  are  not 
limited  to  a  physical  space. 
Collaboratories  can,  in  fact,  contain 
many  independent  laboratories  which 
are  not  collocated,  but  are  connected 
electronically.  The  use  of  these  net¬ 
worked  relationships  makes  possible 
the  effectiveness  of  this  working  group 
to  initiate  and  continue  electronic  com¬ 
munication  to  support  the  objectives 
of  this  group.  • 

Jennifer  Whitestone  is  a  Biomedical  Engi¬ 
neer  with  the  Design  Technology  Branch, 
Human  Engineering  Division,  Armstrong 
Laboratory,  Wright-Patterson  Air  Force 
Base,  OH. 
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INFORUM® 

Data  Collection  and  Consensus  Building  Groupware  System 


Leo  J.  D.  Bernier 


ONFORUM  is  a  revolution¬ 
ary  groupware  system  that 
improves  the  process  of 
collecting  information  and  forming 
consensus.  As  mundane  as  this  activity 
may  sound,  it  is  fundamental  to  the 
way  we  conduct  business  today.  It  has 
been  estimated  that,  in  the  Depart¬ 
ment  of  Defense  (DoD)  alone,  such 
activities  carry  a  price  tag  of  more  than 
$6  billion  annually! 

These  numbers  are  in  fact  conserva¬ 
tive,  especially  as  we  begin  to  account 
for  the  industrial  base  that  supports 
the  DoD.  Consider,  for  example,  end- 
users  of  new  and  existing  systems. 
These  individuals  (and  there  are  many 
of  them)  spend  a  considerable 
amount  of  time  collecting  information 
and  forming  consensus  on  the 
intended  purpose  of  their  systems, 
the  goals  of  these  systems,  and  the 
requirements  of  the  systems.  End-us¬ 
ers  are  also  involved  in  communicat¬ 
ing  these  requirements  to  system  de¬ 
signers  and  in  assessing  the  design 
and  implementation  against  the  origi¬ 
nal  set  of  requirements. 

Human  factors  personnel  are  an 
important  class  of  experts  who  must 
interact  with  end  users.  These  experts 
spend  a  significant  amount  of  time 
setting  human  resource  goals  that  sup¬ 
port  end-user  goals  and  in  refining 
(from  a  human  resource  perspective) 
the  broad  requirements  provided  by 
the  end-users.  These  experts  also  par¬ 
ticipate  in  communicating  these  more 
detailed  requirements  to  the  system 
implementors.  In  addition,  human 
factors  experts  may  interact  with  hu¬ 
man  resource  specialists  to  prepare 
a  human  resource  capability  that 
can  effectively  support  the  system 
being  developed  or  modified.  Once 
again,  in  performing  these  activities, 
considerable  time  and  effort  are  in- 


®INFORUM  is  a  registered  trademark  of 
Bernier  &  Associates,  Inc.  All  rights  reserved. 


volved  in  collecting  information  and 
forming  consensus. 

Not  only  are  such  activities  perva¬ 
sive  in  business  today,  methods  and 
tools  needed  to  support  these  activi¬ 
ties  are  only  now  beginning  to  appear 
in  the  marketplace.  INFORUM  is  one 
such  tool. 

Like  other  automated  collection  and 
consensus  building  systems,  INFORUM 
replaces  the  more  conventional  forms 
of  collecting  information.  These  in¬ 
clude  questionnaires  that  are  adminis¬ 
tered  manually,  live  interviews,  and 
group  meetings.  Each  of  these  meth¬ 
ods  has  unique  strengths  and  weak¬ 
nesses,  but  all  share  the  common 
weakness  of  being  inconvenient  and 
costly  to  implement.  Recognizing  the 
opportunity  to  improve  this  situation, 
software  developers  have  begun  to 
provide  automated  alternatives. 

INFORUM  is  unique  in  that  it  offers 
automated  alternatives  to  all  three 
manual  methods  cited.  It  can  be 
configured  and  used  in  live  data 


collection  sessions  with  either  groups 
or  individuals,  it  can  support  data 
collection  activities  via  a  local  area 
network,  and  it  can  manage  the  col¬ 
lection  of  information  from  individu¬ 
als  at  remote  sites  by  means  of  its 
Electronic  Questionnaire. 

One  way  INFORUM  distinguishes 
itself  from  other  similar  tools  in  the 
marketplace  is  that  it  can  be  used  as  a 
different  time/different  place  system 
(see  Fig.  1).  This  means  that  individu¬ 
als  participating  in  a  data  collection 
and  consensus  building  activity  need 
not  be  physically  located  in  the  same 
place  and  can  participate  at  different 
times.  Most  other  systems  are  same 
time/same  place.  This  feature  of 
INFORUM  permits  great  flexibility  in 
scheduling  the  involvement  of  indi¬ 
viduals  and  virtually  eliminates  the 
expense  (and  the  aggravation)  nor¬ 
mally  associated  with  traveling  to  the 
data  collection  site.  The  bottom  line  is 
that  broader  participation  is  possible 
at  a  lower  cost. 


Figure  1.  INFORUM  allows  for  data  collection  in  a  variety  of  locations  at  different  times 
without  travel  to  the  data  collection  sites. 
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Another  feature  that  distinguishes 
INFORUM  is  its  ability  to  collect 
information  from  multiple  view¬ 
points  simultaneously  on  an  interre¬ 
lated  set  of  topics.  For  example,  let’s 
assume  that  the  system’s  functions, 
goals,  and  requirements  are  defined 
by  the  end-users.  INFORUM  could 
easily  be  configured  to  collect  this 
information  from  all  the  end  users 
and  to  form  consensus  among  them. 
Next,  assume  that  a  design  team  rep¬ 
resenting  different  functional  areas, 
such  as  human  factors,  wishes  to  ex¬ 
tend  the  requirements  and  form  an 
initial  specification.  INFORUM  could 
easily  be  configured  to  collect  this 
information.  It  could  also  be  used  to 
correlate  the  information  provided  by 
the  design  team  with  similar  informa¬ 
tion  provided  by  the  end  users. 
INFORUM  could  then  be  used  to  pri¬ 
oritize  the  requirements  and  specifica¬ 
tions  from  the  point  of  view  of  the 
design  team  or  from  the  point  of  view 
of  the  end  users.  Similarly,  INFORUM 
could  be  configured  to  permit  the 
development  team  to  detail  the  speci¬ 
fications  and  form  an  implementation 
plan.  Collecting  and  managing  infor¬ 
mation  in  this  manner  also  permits 


complete  traceability  of  information 
between  the  viewpoints  (end  users, 
the  design  team,  and  the  implementors 
of  systems). 

This  is  just  one  example  of  how 
INFORUM  might  be  applied.  The 
point  to  remember  is  that  INFORUM 
can  be  configured  to  collect  any 
information,  for  any  purpose,  from 
any  combination  of  viewpoints.  This 
group  of  features  sets  INFORUM  apart 
from  other  data  collection  and 
consensus  building  systems  currently 
in  the  marketplace. 

INFORUM  has  also  been  designed 
to  interface  easily  with  existing  appli¬ 
cations.  This  means  that  it  can  be 
used,  for  example,  to  collect  informa¬ 
tion  to  support  any  number  of  existing 
human  factors  computational  models. 
INFORUM  can  also  import  the 
results  generated  by  these  applica¬ 
tions  and  then  use  them  to  support  still 
other  analyses. 

INFORUM  is  currently  being  used  to 
support  a  variety  of  data  collection 
activities,  such  as  collecting  and  man¬ 
aging  suggestions,  user  requirements, 
modeling  information,  user  feedback, 
and  strategic  planning  information. 

In  one  effort,  INFORUM  is  being 


used  to  develop  a  Human  Resource 
Guide  and  Career  Development  Hand¬ 
book  for  a  large  organization.  In  an¬ 
other  effort,  a  software  company  is 
using  it  to  collect  user  reaction  to  its 
product  line.  In  yet  another  effort, 
INFORUM  is  being  used  to  create  a  set 
of  information  standards.  If  there  is  a 
need  to  collect  information  and  form 
consensus,  there  is  probably  an  op¬ 
portunity  to  use  INFORUM. 

INFORUM  is  written  in  CLIPPER 
5.01  and  is  designed  to  run  on  any 
system  that  supports  DOS  3-3  or 
higher.  The  system  is  being  developed 
under  a  Small  Business  Innovation 
Research  Program  initiative  for  the 
Computer  Resource  Technology  Tran¬ 
sition  Program  managed  by  the  Sys¬ 
tems  and  Software  Design  Center,  part 
of  the  Electronic  Systems  Center  of  the 
Air  Force  Materiel  Command. 

For  further  information  about 
INFORUM,  please  contact  Michael 
O’Neill  at  Bernier  &  Associates,  Inc.,  458 
Boston  St.,  Topsfield,  MA  01983.  Tele¬ 
phone:  (508)  887-8683,  Fax:  (508)  887- 
6592,  E-mail:  ljb@world.std.com  • 

Leo  f.  D,  Bernier  is  President  of  Bernier  & 
Associates,  Topsfield,  MA. 


Editor's  Note:  In  the  last  issue  of  Gateway,  we  inadvertantly  used  an  out -of -date  photograph  in  the  SCOPE  ad  (below).  We  have  corrected 
that  and  offer  our  apologies  to  the  U.S.  Army  Research  Laboratory.  JAL 


■  SCOPE  IT  OUT!  ■ 

Smart  Contract  Preparation  Environment  from  the  U.S.  Army  Research  Laboratory 


wo  software  programs  intended  to  assist  in 
the  request  for  proposal  (RFP): 


the  preparation  of 


SCOPE  (Smart  Contract  Preparation  Environment) 


SPEC  Maker  -  to  facilitate  the  tailoring  of  two  commonly  used  HFE 
design  standards  (MIL-STD-  1472D,  Human  Engineering  Design  Criteria 
for  Military  Systems,  Equipment,  and  Facilities,  and  MIL-STD- 1 474C, 
Noise  Limits  for  Military  Materiel)  for  incorporation  in  the  system 
specifications. 

CDRL  Maker  -  to  facilitate  the  preparation  of  DD  Forms  1423,  the 
Contract  Data  Requirements  List,  and  the  tailoring  of  the  human 
engineering  Data  Item  Descriptions  (DIDs),  DD  Forms  1664. 

Price:  $35  each.  For  further  information  on  SCOPE  and  its  Wo 
products,  SPEC  Maker  and  CDRL  Maker,  contact  CSERIAC  at  (513) 
255-4842. 
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CSERIAC 
PRODUCTS  AND 
SERVICES 

CSERIAC's  objective  is  to  acquire, 
analyze,  and  disseminate  timely  infor¬ 
mation  on  crew  system  ergonomics 
(CSE).  The  domain  of  CSE  includes 
scientific  and  technical  knowledge  and 
data  concerning  human  characteris¬ 
tics,  abilities,  limitations,  physiological 
needs,  performance,  body  dimensions, 
biomechanical  dynamics,  strength,  and 
tolerances.  It  also  encompasses  engi¬ 
neering  and  design  data  concerning 
equipment  intended  to  be  used,  oper¬ 
ated,  or  controlled  by  crew  members. 

CSERIAC's  principal  products  and 
services  include: 

■  technical  advice  and  assistance; 


■  customized  responses  to  biblio¬ 
graphic  inquiries; 

■  written  reviews  and  analyses  in 
the  form  of  state-of-the-art  reports  and 
technology  assessments; 

■  reference  resources  such  as  hand¬ 
books  and  data  books. 

Within  its  established  scope,  CSERIAC 
also: 

■  organizes  and  conducts  work¬ 
shops,  conferences,  symposia,  and 
short  courses; 

■  manages  the  transfer  of  techno¬ 
logical  products  between  developers 
and  users; 

■  performs  special  studies  or  tasks. 

Services  are  provided  on  a  cost- 
recovery  basis.  An  initial  inquiry  to 
determine  available  data  can  be  ac¬ 
commodated  at  no  charge.  Special 
tasks  require  approval  by  the  Govern¬ 
ment  Technical  Manager. 


To  obtain  further  information  or  re¬ 
quest  services,  contact: 

CSERIAC  Program  Office 
AL/CFH/CSERIAC  Bldg  248 
2255  H  Street 

Wright-Patterson  AFB  OH  45433-7022 

Telephone . (513)  255-4842 

DSN . 785-4842 

Facsimile . (513)  255-4823 

Government 

Technical  Manager . (513)  255-8821 

Director:  Mr.  Don  A.  Dreesbach; 
Government  Technical  Manager:  Dr. 
Reuben  L.  Hann;  Associate  Govern¬ 
ment  Technical  Manager:  Ms.  Tanya 
Ellifritt;  Government  Technical  Director: 
Dr.  Kenneth  R.  Boff. 


CSERIAC  Gateway  is  published  bi¬ 
monthly  and  distributed  free  of  charge  by 
the  Crew  System  Ergonomics  Information 
Analysis  Center  (CSERIAC).  Editor:  Jeffrey 
A.  Landis;  Copy  Editor:  R.  Anita  Cochran; 
Illustrators:  Ronald  T.  Acklin,  Timothy  J. 
Span;  Layout  Artist:  Vicky  L.  Chambers;  Ad 
Designer :•  Kristen  Cheevers. 
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